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SUMMARY 

\ 

A  coaputatlonal  procedux*e  is  given  for 
finding  the  minimum  of  a  quadratic  function 
of  variables  subject  to  linear  inequality 
eonatrainta.  The  procedure  is  analogous 
to  the  Simplex  Method  for  linear  programming, 
being  based  on  the  Barankin-Dorfman  procedure 
for  this  problem. 
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THB  SZNFLBX  METHOD  FOK  Qf  ASIATIC  PROORAWIXIIO 
1.  lETHODPCnOW 

In  this  papsr^  by  "qusdrstlo  progrsMdng"  ws  shall 
undsrstand  ths  problsm  of  dstsrmining  valuss  of  ssvtral  rsal 
variablss,  subjsot  to  llnsar  insquality  constraints,  which 
yield  ths  sxtrsas  value  of  a  quadratic  function.  Besides 
being  a  step  on  the  way  toward  solution  of  the  elaborate  non¬ 
linear  prograaning  problems  which  economic  models  often  present, 
a  usable  computational  procedure  for  quadratic  programming  can 
be  applied  to  a  number  of  problems  of  interest  in  themselves: 

Regression .  To  find  the  best  least -squares  fit  to  given  data, 
where  certain  parameters  are  known  a  priori  to  satisfy 
inequality  constraints  (e.g.,  being  non-negative) • 

Efficient  production.  Maximization  of  profit,  assuming  linear 
production  functions  and  linearly  varying  marginal  costs. 

"Portfolio"  problem.  To  find  a  combination  of  random 
variables  having  given  expectation  and  minimum  variance. 

Convex  prograiidng.  To  find  the  minimum  of  a  general  convex 
function  under  linear  constraints  using  a  quadratic  approximation. 

Let  the  variables  of  the  problem  constitute  the  n -vector 
X  ■  (x^,  Xj^)*  (*  will  denote  transposition;  we  take  x  to 

be  a  column  vector,  that  is,  an  n-^y-l  matrix).  Letting  A  be 

*ttue  io  Warkowitz  R>]  • 


an  m  bx  n  natrix,  and  b  m^by-1,  «•  will  axpraaa  tha  llntar 
conatrainta  of  tha  problan  by 

(1)  X  2  0,  Ax  -  b, 
that  la, 

Xj  i  0  (>1.  ...  n),  a^jXj  -  b^  (1-1,  ...  m). 

Vt  raoall  that  any  eoabination  of  linaar  aquation  and 
inaquality  conatrainta  can  ba  writtan  in  tha  fona  (1)  if 
additional  variablaa  ara  allowad  into  tha  problam. 

Lat  p  ba  a  1-by-n  natrix,  and  C  ba  an  n-by-n  aymatrio 
matrix.  Wa  ahall  writa  tha  objaotiva  function,  tha  (partly) 
quadratic  function  to  ba  axtramizad  aubjact  to  (1),  aa 

(2)  t(X,x)  m  ^px  +  1/2  x'Cx, 
or 

-aIj 

whara  ^  ,  a  ainglt  non-nagativa  raal  paramatar,  can  ba  ohoaan 
at  oonvaniant.  Tha  problaa  can  now  ba  atatad  aa: 

The  quadratic  problaa  for  X  ^  ^  * 

(3)  lUniaiza  f(/(^,x)  •  Apx  +  l/2  x*Cx 
subject  to  X  ^  0,  Ax  ■  b. 

An  important  raatriction  nuat  ba  placed  on  tha  quadratic 
part,  C,  of  tha  objaotiva  function  in  order  to  anaura  tha 
auaoass  of  tha  computational  aathod:  tha  function  f  auat  ba 
convex,  that  ia,  C  nuat  ba  poaitiva  aamidaf inita .  This 
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condition  -  apparently  essential.  In  the  present  state  of  the 
art,  to  all  non-linear  programming  schemes  -  ensures  that  any 
local  minimum  encountered  in  the  problem  will  be  the  global 

solution  desired.  Algebraically,  the  assertion  of  positive 

\ 

semidefiniteness  for  C  is  that 
(k)  x*Cx  >  0  for  all  x. 

In  economic  problems,  it  is  the  ascription  of  non-increasing 
returns  to  scale  for  all  activities,  since  the  marginal  cost 
of  changing  from  the  program  x  to  the  program  x  +  Ax  is  given  by 

^  f(X,x+tAx)  ■  pAx  +  xCAx  +  tAx'CAx, 

which  will  be  a  non -decreasing  function  of  t.  We  shall  assume 
it  from  now  on.  A  more  detailed  discussion  of  the  role  of 
this  property  in  quadratic  programming  is  given  in  [4] , 

A  number  of  proposals  for  the  computational  solution  of 
quadratic  programming  problems  have  been  made  In  the  last  two 
years;  those  which  seem  suited  to  high-speed  digital  computers 
are  given  in  References  [l]  -  [6]  below,  Barankin  and  Dorfman 
[1]  first  pointed  out  the  linear  formulation  of  the  quadratic 
problem,  Inspiring  the  present  approach;  our  Section  2  Is  taken, 
with  changed  notation,  from  Sections  1  and  3  of  [l] . 

The  principal  respect  in  which  the  present  method  differs 
from  these  Is  In  using  only  the  computational  mechanisms  of  the 
Simplex  Method  for  linear  programming  problems.  It  Is  there¬ 
fore  a  simple  matter  to  convert  a  computing  machine  code  for 
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llnear  progranning  Into  one  for  quadratic  programming;  the 

SHARE  linear  programming  code  for  the  IBM  704  requires 

modification  in  eleven  Instructions  for  this  purpose. 

In  the  sequel  this  method  is  developed  in  two  forms, 

"short"  and  "long."  The  computations  of  the  long  form  are 

like  those  of  the  short,  but  are  aimed  at  solving  the 

quadratic  problem  (3)  for  all  >  >  0  and  at  the  same  time 

avoiding  certain  restrictions  on  the  use  of  the  short  form. 

The  table  below  summarizes  the  conditions  for  use  of  these 

« 

methods.  The  estir/iate  of  the  number  of  Simplex  Method  change s- 
of-basls  needed  to  solve  the  problem  is  based  on  experience  like 
that  described  in  Section  6. 

Solution  of  (3)  by  - 


Conditions 


Solution 
obtained  for 

Size  of  equi¬ 
valent  linear 
program 


Bstimated  number 
of  basis  changes 
for  solution 


N 


Sbart  fvm  iMt  frm 


Either  A  •  0  or  C 
positive  definite 

C  positive  semidefinite 

fixed  A 

•u  A  i  0 1 

1  m  -  .  '  -  ^  1 

up  to  m't'n  equations,; 
m4'3n  variables  / 

!up  to  »fn  equations,; 
j  »^3rH"l  variables  j 

2(*-w) 

i 


Sinc«  w«  art  Intaraattd,  in  part«  in  tha  solution  of  tht 
quadratic  problta  for  all  0,  lat  ua  dafina 

for  0i  P(A)  •  Min^^px  ♦  1/2  x'cx  tx^O,  Ax  ■  b}* 

Qulta  a  bit  of  information  about  F(a)  can  ba  obtained 
without  calculating  it«  Throughout  it  will  ba  assumed,  of 
course,  that  there  exist  feasible  x,  i.e.  x  ^  0  such  that  I 

Ax  ■  b.  Nevertheless,  wa  may  have  F(A)  •  -go  for  some  j 

(a^  hence  all)  A>  0. 

First,  an  important  feature  of  the  positive  semi- 
definiteness  of  C  Is  given  by 

LINNA  li  x'cx  -  0  isDlias  Cx  -  0. 

-  i 

Proof:  For  any  n-vector  y  we  have  i 

0  ^  (yftx)  *C(yttx)  •  y  Cy  ♦  2ty*Cx  for  any  number  t,  whence  ! 

I 

y  Cx  *  0|  Cx  •  0  follows  at  once. 


This  leads  to  I 

UMMA  2i  For  any  C,  the  solution  set  of  all  feasible 
X  such  that  f(A,x)  ■  F(A)  Is  the  Intersection  of  a  linear 

! 

manifold  with  the  constraint  set,  and  px  Is  constant  on 
this  set  for  A>  0. 

I 

Proof  I  Let  feasible  points  x,  y  be  given  such  that  ; 
#*)  •  ^‘(A,  y)  -  F(a)  .  Letting  w  -  y-x,  for  any  0^t£l 
the  point  x4>tw  belongs  to  the  constraint  set;  since  f  Is 


f 

P-1205 
^  10-25-57 

-6- 


conv«x,  and  f  (  A  »  x4>tw)  la  nlnlattl  f  or  t  ■  0  and  t  ■  1, 
wa  havt  f(  A,  x-ftw)  •  f(A  ,  x)  for  all  0  <t  ^  1,  or 
Ap(x-»-tw)  1/2  (x+tw)  *C{x*i'tw)  •  Apx  ♦  1/2  x*Cx,  which 
almpllflta  to 

(  X  pw  -f  x*Cw)t  1/2  w*Cw  •  0  [0;;^t£l]  • 

I 

Thua  w  Cw  •  0,  whence  by  Imm  1 


few  ■  0,  and  hanoa  I 

pw  -  0. 

Convaraaly,  It  la  clear  that  If  f(X  >x)  ■  F(X),  w  aatlafiaa 
(5)#  and  x  tw  la  feaalble,  then  f(A,  x^tw)  •  F(A)i  bo 
that  the  conpleta  solution  sat  for  A  glvan  is  tha  Inter-  j 

I 

section  of  tha  constraint  set  with  tha  linear  nanlfold  j 

{  X  4  twj  «  Iquation  (5)  yields  finally  px  ■  py  for  any  | 

two  solutions.  ! 


i 


If  now  for  any  X  >  0  we  choose  a  feasible  solution  x^  ' 

I 

such  that  f(  A  ,x^  )  •  F(A),  by  leana  2  the  value  px^  Is  | 
Independent  of  tha  choice  of  X;v  •  ! 

i 

I 

Theorem  1:  For  A  2  0#  F(a)  Is  a  concave  function;  px^^  la  | 

^  ^  * 

monotone  non-lncraasing;  and  x^^  is  a  solutloni^y  of  t)M  I 

problem!  | 

Min  [y*Cy  :  y>p.  Ay  •  b,  py  £  px^  ^  .  ( 

Proof:  Since  f (  A  »  a)  Is  linear  In  X  ,  tha  function  ! 

i 

F(X)  Is  tha  infl^wp  of  a  family  of  linear  functions,  and 
hanoa  concave.  ~ 
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Tht  trend  of  px,  ie  en  inetence  of  a  quite  general  faetl 

^  i 

Take  any  ^  and  u  •  Since  minlmiaea  f(^«x)«  we  have 

'  j  \  , 

♦  1/2  Cx^  ^  4  1/2  CXyL,  ;  > 

and  ainee  x^  miniaizee  we  have 

/*px^  ♦  1/2  y  <  yopx^  -f  1/2  xJ^CXj;^  . 

Adding  thaae  ineq\ialitiet  and  rearranging,  we  get 

^-;\)pj^  ^  ^-7l)px^  , 

which  yields  pj^  px^  for  >  ^  • 

Finally,  since  X;^  does  minimize  Apx  +  1/2  x  Cx,  any 

I  I 

y  such  that  y  Cy<X;^Cx^will  give  py>px^,  which  proves  the 
last  statement. 

The  next  theorem  characterizes  x^  in  such  a  way  that 
we  will  be  able  to  compute  it.  Only  the  sufficiency  of 
this  condition  for  the  minimization  of  f(h,x)  is  needed,  since 
its  necessity  will  follow  when  we  have  established  that 
results  of  the  computational  scheme  of  the  next  section 
meet  this  condition  if  the  minimum  exists. 

THEORIM  2 1  If  x>0,  Ax  ■  b,  and  there  exist  v^O  (v  is  n  by  l) 
and  u  (u  is  m  by  l)  such  that 

(6)  v*x  ■  0 

(7)  Cx  -  V  a'u  ^p*  ■  0, 


and 
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then  X  aolvtt  the  problem  Min^  Apx  4>  1/2  x  Cx  :  x>0.  Ax  •  b  J  | 

1 

Proof:  Let  any  y^O,  Ay  •  b  be  given.  We  shall  show  that 
f(l#y)  >  f(>,x).  From  the  positive  semi -definiteness  of  C 
we  have 

(y-x) ’  C(y-x)  >  0, 
whence  ^  ^  2x'cy 

I 

y'cy  -  x*Cx  2  2x*C(y-x) 

80  that 

f(X,y)  -  f(X,x)  -  >p(y-x)  l/2y*Cy  -  1/2  x'cx  • 

I 

>  (Xp^x'c)(y-x). 

Since  by  (?)  , 

I  I  •  j 

Ap+x  C  -  V  -u  A,  I 

I 

f(X,y)  -  f(X,x)  >  v'y  -v'x-  u'Ay  +  u'ax  i 


-  v*y  -  0  -  u«b  ♦  u*b  (by  (6)  and  feasibility) 
•  V  y  2  0  (since  v,y  2  •  ' 

The  conditions  (6)  and  (7)  -  especially  as  nece^ary,  ' 
rather  than  sufficient,  conditions  —  are  essentially  those  ^ 
of  the  "saddlepolnt"  theorem  of  Kuhn  and  Tucker  [9].  In  the 
present  form,  the  result  Is  due  to  Barankln  and  Dorfman  [l. 

Section  3].  The  theorem  In  fact  obtains  If  f(X,x)  Is  replaced 
by  any  convex,  differentiable  function  and  Cx  +  Xp  of  (7)  Is 
replaced  by  Its  gradient. 
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The  reaarkAble  feature  of  the  quadratic  problem  la  the 
linearity  of  the  gradient  of  which  confines  the 

non-llnearlty  of  the  Kuhn-Tucker  conditions  to  the  relation 
(6)«  v'x  ■  0,  which  has  this  sort  of  combinatorial  expression! 

(8)  Vj  >  0  implies  ^  (J*l«**#n). 

In  order  to  explore  the  relation  of  the  constraints  (8)  < 
to  the  linear  relations  (j) ,  and  see  how  they  may  be  handled 
numerically,  we  must  consider  briefly  the  main  features  of 
the  Simplex  Method  for  linear  programming  [7J  . 

It  is  required  to  minimize  the  linear  form  cx  under  the 
constraints  I>x«e,  x>0  (D  is  p  by  q,  c  is  1  by  q,  e  is  p  by  l). 

We  suppose  this  problem  feasible,  l.e.,  the  existence  of  x 
satisfying  the  constraints.  It  is  easy  to  show  through 
linear  dependence  that  there  exists  a  feasible  x  having  no 
■ore  than  p  positive  components.  A  collection  of  p  columns 
froa  D  which  correspond  to  the  non-vanlshlng  components  of 
a  feasible  x  is  called  a  basis,  and  x  Is  called  a  basic 
solution.  In  the  Simplex  Method  one  works  always  with  such 
bases;  given  any,  it  is  shown  that  either  (i)  the  associated 
basic  solution  yields  the  minimum  value  of  the  linear  form, 
or  (ii)  another  basis  differing  in  only  one  column  from  the 
given  basis  can  be  found  whose  associated  basic  solution 
yields  a  sKaller  value  for  the  form,  or  (ill)  one  coluiui  can 
be  adjoined  to  the  basis  such  that  a  sequence  of  feasible  x's 
associated  with  these  p^l  columns  can  be  found  on  which  cx->  -00. 
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ThU8  a  sequence  of  bates  it  generated  which  terminates  in 
either  a  finite  or  infinite  solution  of  the  problem.  It  is 
convenient  to  make  an  assumption  of  "non-degeneracy'*  regarding 
the  constraints  of  the  problem:  That  any  feasible  vector  x 
has  at  least  p  positive  components.  A  consequence  of  this 
assumption  is  the  linear  Independence  of  the  columns  of  any 
basis.  It  has  been  shown  [7]  that  every  set  of  constraints 
can  be  dealt  with  so  as  to  be  non-degenerate.  In  the  sequel 
we  rely  on  these  results,  assuming  non -degeneracy  in  the  few 
places  it  is  necessary. 


Returning  to  the  quadratic  problem,  the  conditions  that 
the  n-vector  x  solve  the  quadratic  problem  for  X>p  may  be 
written  together  as  (omitting  for  the  moment  v  x  •  O) 

Ax  •  b, 

Cx  -  V  +  A'u  +p>^  .  0, 

X  2  Of  V  2  0, 

or  in  detached  coefficient  form  as 


(9)  u  \ 

A  0  0  0 

1 

,C  -I  A'  p» 


constituting  m  -f  n  equations  in  2n  non-negative  variables  and’ 
m  unrestricted  variables  (A  is  not  considered  a  variable) . 

We  will  be  concerned  below  with  the  basic  solutions  of  this 
system.  Note  that  the  ro  coIusdis  associated  with  the 


I 
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i  unr«ftrict«d  variablt  u  ara  taken  to  be  In  every  basis  (this  \ 

i 

technical  device  makes  it  unnecessary  algebraically  to 
eliminate  the  u's  to  bring  the  system  to  standard  fom)^ 
leaving  only  n  positive  variables  in  any  basic  solution  to  < 
go  to  X  and  V. 

Assuming  for  the  moment  the  converse  of  Theorem  2,  if 
the  quadratic  problem  has  a  solution,  then  there  exist  x,v,u 
satisfying  (6)  and  (7) •  But  (6^  implies  that  at  least  n 
components  from  the  2n  x  and  v  vanish;  and  this  establishes 
the  important  result  of  Barankin  and  Dorfaan  [l]  that  sosm 
basic  solution  of  (9)  constitutes  a  solution  of  the  quadratic 
problem. 

Since  the  computational  step  of  the  Simplex  Method  can 
be  used  to  explore  basic  solutions,  Barankin  and  Dorfman 
have  accordingly  proposed  use  of  the  method,  beginning  with  an 
arbitrary  basic  solution  of  (9)#  to  reduce  v*x  to  zero*  One  method 
which  accomplishes  this  is  given  in  [4],  but  it  is  more  com¬ 
plicated,  and  probably  slower,  than  the  present  algorithm. 

Markowitz,  on  the  other  hand,  has  suggested  a  method  [6] 
for  the  "portfolio**  problem  (equivalent  to  solving  the 
quadratic  problem  for  all  A>0)  which  begins  with  constraints 
looser  than  (9)  and  which,  while  retaining  (8)  v'x  -  0,  should 
alter  the  variables  until  (9)  obtains.  The  method  described 
here  exploits  this  ingenious  idea,  differing  from  the  proposal 
of  [6]  in  keeping  to  a  linear  programming  format. 
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3.  THI  COWPfTATION 

Htrt  «•  prtttnt  tht  ooBputatlonAl  algorithas  for  tho 
minimization  of  Apx  -f  1/2  x*  Cx  aubjaot  to  x^O^  Ax«  b. 

First  is  given  the  "short  form, "  for  >  fixed,  whose  con¬ 
vergence  requires  that  either  A  ■  0  or  that  C  be  positive 
definite;  next  is  given  the  "long  form,"  solving  the  quadratic 
problem  "parametrically"  for  all  A  ^0,  which  does  not  need 
C  positive  definite,  but  which  involves  two  recursions  of  the 
"short  form"  type* 

We  will  suppose  below,  relying  on  j^?]  $  that  the  con¬ 
straints  Ax  «  b  and  the  constraints  employed  below  are  all  non- 
degenerate • 

SHORT  PORW 

Let  z  ,  z^,  and  w  be  respectively  n-,  n-,  and  m— coi^ponent 
vectors.  We  begin  with  the  set  of  relations 

(10)  Ax  w  •  b 

(11)  Cx  -  V  +  A*u  -f  z^  -  z^  •  -Ap^ 

(12)  X,  V,  z^,  z®,  w  ^  0, 
a  weakening  of  the  set  (9)« 

Initiation 

Since  b  ^  0,  an  initial  basis  for  this  system  can  be 

1  2 

formed  from  the  coefficients  of  z  ,  z  ,  and  w.  fee  the  simplex 


method  to  minimize 
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(13)  Wi 

to  ztro,  k««ping  v  tnd  u  z«ro.  Diaeard  m  and  tha  unused 
components  of  z  ,  z^;  let  the  remaining  ;n|eo«ponents  be 
denoted  by  Z,  and  their  coefficients  by  E.  We  now  have  a 
solution  of  the  system 

(14)  AX  -  b 

Cx  -  V  -f  A*u  ^  EZ  m  -y(p' 

X,  y,  Z  ^  0. 

Recursion 

Qiven  a  basis  and  basic  solution  satisfying  (14),  (8) 
n 

v*x  ■  0,  and  Zj  >  0,  make  one  change  of  basis  in  the 
Simplex  procedure  for  minimising  the  linear  form 

L  *«• 

under  the  side  condition 

(16)  for  k  ■  :  if  is  in  the  basis,  do  not 

admit  v^;  and  if  is  in  the  basis,  do  not  admit 

^k- 

Termination 

If  the  form  (15)  is  positive,  repeat  the  recursive  step. 
The  form  will  vanish  in  at  most  (^)  iterations,  yielding 
Z  ■  0.  The  x-part  of  the  terminal  basic  solution  is  a 
solution  of  the  quadratic  programming  problem  for  A  . 


L(»<0  FORM 


Initiation 

Having  parfomad  tha  abort  fom  eoaputation  for  A*  0, 
add  p*  to  tha  short  fom  data,  obtaining  tha  ayataa 

(17)  Ax  -  b 

(18)  Cx  -  V  +  A*u  -f  mP'  ♦  BZ  -  0 

and  an  initial  solution  having  u  ■  0,  Z  ■  0,  and  v*x  ■  0. 
Racuraion 

Oivan  a  basis  and  basic  solution  satisfying  (17)#  (18), 
v*x  •  0,  and  having  Z  •  0,  aaka  ona  ohanga  of  basis  (if 
possibla)  in  tha  Sinplax  procadura  for  ainiaizing  tha  linaar 
fom 

(19)  -n 

undar  tha  sida  condition  (16)  and 

(20)  do  not  allow  any  Zj  in  tha  basis. 

Tamination 

If  it  is  not  possibla  to  aaka  tha  basis  ohanga  of  tha 
raoursion,  than  m.  *  0,  F(A)  •  —  co  for  allX>  0,  and  a  sat  of 
faasibla  x  can  be  found  (in  Saotion  3)  on  which  f(^  ,x)— > -oo 

forA>  0. 

Otharwisa  tha  raeursion  will  yiald  tha  finita  saquanoa 
of  valuas  0  ■  u®  <  <  ••  <  and  tha  x-parts  x®,x^,..,x*^ 
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of  their  aseoelated  baeio  solutions,  terminating  in  at  most 

(5")  iterations  with  the  vector  ,  such  that: 

'n 

ic.  1(^1 

for  M>  ^  quadratic  problem  for  A  la  solved 

by 


jr 

and  for  A  ^  it  is  solved  by 
(22)  X  -  (A  X®  . 

NOTli  B.  M.  L.  Beale  recently  communicated  an  elegant  modi¬ 
fication  of  the  "short  form"  procedure  above  which  permits 
its  use  in  the  case  that  the  quadratic  form  is  only  positive 
semidefinite  instead  of  definite.  It  consists  essentially  in 
calculating  the  effect  of  a  "virtual  perturbation*  of  C  which 
involves  replacing  Cjj  by  J  •  l,..,n,  for  arbitrarily 

small  6,  so  that  the  algorithm  can  operate  as  if  a  positive 
definite  form  were  esiployed. 
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4.  PAMPLK 

At  an  txaiQ)!*  for  calculating  with  both  tha  short  fora 
and  tha  long  fora,  wt  shall  solva  ths  problsa. 

Min  ^  K 

subjact  to  x^#X2,x^,  x^-Xg-^x^  -  1.  j 

j 

The  objactlva  function  can  ba  written  | 

I 

f(\,x)  -  l/2[(Xj^)g*+Xj“-t-  (*3-*>^)*]  -  5/8  X*. 

and  thus  for  any  X  the  solution  x  will  ba  that  point  of  ! 

tha  constraint  sat  closest  to  tha  point  (—A*  0,  2X).  This  | 
is  illustrated  in  Pi^^ura  1  for  I,  and  in  Figure  2  for  j 

general  A>P.  j 

For  this  problaa  I 


« 

I 

I 

I 

; 

I 

I 

I 

1 
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PiaURE  1;  SHORT  FORM 


r-  ^ 


f 
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0  0 
1  0 
0  1 

• 

P  -  [l  0  -s] 

since  C  is  positive  definite,  the  8hox*t  fom  eill  solve  the 
problem  for  any  A  •  Taking  A  -  1,  foxmiae  (10,11)  give  the 
Initial  array  tor  the  short  fofVL. 


-  1 

-  -1 

•  0 
•  2 


Although  there  is  considerable  degeneracy  In  the  problem,  the 
minimization  of  proceeds  without  hitch.  Below  are 
given  the  values  of  the  variables  In  the  successive  steps 
(only  the  values  of  the  basic  variables  are  given).  The 
variable  u  is  introduced  first,  since  it  will  be  in  every 
basis.  Since  it  is  unrestricted,  it  might  have  been  eliminated 
from  the  system,  but  we  have  left  it  in. 


1-11 


1 

0 

0 


0  0 
1  0 
0  1 


0  0  0 

-10  0 
0-10 
0  0-1 


-1 

1 


0  0  0 

10  0 
0  10 
0  0  1 


0  0  0 

-10  0 
0-10 
0  0-1 


0 

0 

0 
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Basis 

0 

1 

2 

3 

4 

5 


\ 


1 


7  I 

\  I 


I 


2 

2 

1 

1 

7 

1 

7 


M 

1 

1 


Zz 

^  Initia 

6 

3 


I 


0 


The  path  traced  out  by  x—  from  (1,0,0)  to  (0,0,1)  to  (O,^,^)  — 
is  sketched  in  Figure  1. 

For  the  long  form,  the  initial  array  for  the  problem  is 


*1 

X2 

"^3 

''1 

'^2 

''3 

u 

4 

"2 

4 

4 

w 

1 

-1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

c 

0 

0 

1 

1 

0 

0 

-1 

0 

0 

1 

1 

1 

0 

0 

-1 

0 

0 

0 

0 

1 

0 

0 

-1 

0 

-1 

0 

0 

1 

0 

0 

-1 

0 

0 

0 

0 

1 

0 

0 

-1 

1 

-2 

0 

0 

1 

0 

0 

-1 

0 

1 

0 

0 

0 


The  sequences  ef  values  are : 


\ 

% 
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Basii 


w 


U1 


0 


0  0  0 


1 

2  0 


0  0  0 
0  0 


3 

4 

5 

6 

7 

8 


0 

▲ 

1 

7 


0 

1 

z 

1 

I 

1 


1 


t  1-ft 


.00 


(0,1,1) 


1 

1 


1 

I 


0 

1 

"7 

1 

'7 


1  1 

“I  I 


1 


short 

form 


The  x-parts  of  these  solutions  are  traced  out  in  Figure  2t 


\ 
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^ A  tT’i 


PIOURE  2:  LONG  FORM 
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For  •xjuipltf  tht  solution  of  tht  problta  for  A  ■  l/k 
is  givtn  by  (21)  as  an  interpolation  batween  the  Basis  3  And 
Basis  6  solutions  as  follows: 

*  "  5^^  ^ 

The  solution  here  for  A-  1  is  given  direotly  from  Basis  8 
(of.  formula  22)  for  t  ■  1/2  as  (0,1/2, 3/2) >  the  same  answer 
the  short  fon  gave. 


-  j 
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Tht  burden  of  this  Section  la  to  prove  the  statements  i 

I 

regarding  the  two  terminations  of  the  recursions  of  Section  3* 
The  initiation  of  the  process  (v-O)  and  the  side  condition 
(id)  on  the  choice  of  x's  and  v's  entering  the  basis  have, 
been  designed  to  ensure  that  at  each  stage  of  both  recursionSj 

I 

we ) have  v  x*0.  It  remains  to  see  what  occurs  when,  in  either 
recursion,  it  is  not  possible  to  continue  the  indicated 
minimization  under  the  side  conditions.  The  theorem  below 
gives  what  is  needed  for  analysis  of  these  conditions. 


THEOREM 


Let  A,  b,  C  be  as  in  Section  1;  let  the  matrix  Q  be  n 

(  t 

by  n  ,  q  be  1  by  n  ,  and  g  be  n  by  1.  Let  x>0,  v>0  such  that 

t 

V  x«0  be  given.  Denote  by  x  those  components  of  x  which  are 
positive,  and  by  v^  the  corresponding  components  of  v 
(note  v  "O) ;  denote  by  v  the  positive  components  of  v,  and 
by  Xy  the  corresponding  components  of  x  (note  Xy-O) . 

If  the  linear  form 


(23)  qw 


is  minimal  under  the  linear  constraints 

(24)  V,  -  0 

x^  •  0  and 


(25)  Ax  -b 

Cx  -Iv  -fA  u  -t^w  -g. 
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coefficltnts  appear  below  the  array.  The  columns  correspondlhg 
to  x^*0,  v^«0  can  be  die regarded >  since  we  Insist  that  these  , 
variables  vanish.  ' 

At  the  left  of  the  array  stand  the  variables  of  the  I 

linear  programsing  problem  dual  to  ours  [6] .  The  coefficients 

i 

of  the  linear  fom  for  the  dual  problem  are  those  on  the  ^ 

right  of  the  array;  the  constraints  for  the  dual  problem,  | 

read  vertically  from  the  matrix,  have  their  constant  | 

\ 

coefficients  on  the  bottom  of  the  array.  The  dual  variables 

I 

are  unrestricted  in  sign  since  they  are  connected  with 
equation  constraints.  The  existence  of  these  variables,  ' 

satisfying  the  relations  Indicated  below  the  matrix,  is  the  i 

I 

consequence  of  the  duality  theorem  for  linear  programming  [8]^ 

as  is  the  equality  of  the  two  objective  functions.  (Note 

particularly  that  where  a  variable  turns  out  to  be  non-sero, 

as  in  X  ,  or  is  never  restricted,  as  in  u,  the  corresponding  I 

^  1 

dual  relation  is  an  equality.)  > 


In  detail,  these  relations  are: 


(a) 

'■x  ♦  '•6  °x6  '•v  °xv 

-  0 

(b) 

®*6  '"x  “^xB  '"b^BB  '"v  ®Bv 

£  0 

(c) 

-  '•b  ^6 

<  0 

(27) 

(d) 

—  r  In 

V  V 

-  0 

(*) 

••x  *x  +  *‘b  *8  *  '■v  *v 

-  0 

(f) 

[Vb^'v^  ** 

and 

(e) 

qw  •  sb  “f  rg. 

which  expresses  the  equality  of  the  objectives. 
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‘  -  -  - 

I 

then  there  exlits  r  auch  that 


rC  ■  0,  Ar  •  0,  and  qw  •  rg. 

PROOF:  (Note  that  (2^  la  preclaely  the  linear  expreaalon 

of  the  basis  restriction  side  condition  (l8j)  The  proof 
depends  upon  the  detailed  structure  of  the  quantities  x,  v, 
u,  w  which  yield  the  minimum  of  qw.  We  have  already 
distinguished  In  the  vectors  x,  v  the  corresponding  parts 
X  >0,  V  -0,  and  the  corresponding  parts  x  ■©,  v  X).  There 

4k  V  V 

remain  the  corresponding  parts  x^,  v^,  which,  although  not 
positive,  are  not  required  to  vanish  by  the  constraints. 

In  (26),  below, the  matrix  for  the  constraints  (25)  Is 
partitioned  In  accordance  with  this  partitioning  of  x  and  v, 
first  vertically,  then  horizontally  In  the  naturally 
corresponding  manner  (such  that  -I  partitions  into  diagonal 
matrices) . 


’'x>« 

*6^ 

Xy-O 

< 

X 

1 

o 

V5>0 

V  >0 

V 

u 

w>0 

s 

*6 

A 

V 

0 

0 

0 

0 

0 

{»6)  r. 

®xx 

0 

0 

1 

*x 

^x6 

Sv 

0 

-^6 

0 

1 

*6 

Q 

••v 

Sv 

c 

VV 

0 

0 

1 

*v 

0000000  q 


-b 


According  to  the  hypothesis  of  the  theorem,  the  values  of 
the  variables  above  this  array  minimize  the  linear  font  whose 
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Rtlations  (d)  and  (c)  yiald  at  onca 

I 

ry-O,  rg>p.  i 

Dropping  tharcfort  r^,  multiplying  (a)  on  the  right  by  and 
(b)  by  r^,  wa  have  I 

*  '‘x®xx*'x  ♦ 


which  are  added  to  form 


8 


k-' 

[  X  X 


X  ♦  V#' 


r  r- 
X  6j 


l°x8  °86 


k'-el  ’ 


0, 


By  (e),  howeveri  the  first  term  here  vanishes;  and  the  matrix 
of  C's  in  the  second  term^  being  a  principal  subinatrix  of  a 

t 

positive  semidefinite  matrix  it  itself  positive  semidafinite, ! 

I 

so  that  the  second  term  is  in  fact  zero;  whence,  by  lanma  1, 


0,  or 


(28) 


I 

C  r 

XX  X 
I  i 

°x8‘'x 


I 


•f  c 

+  C 


xO  ^6 


0, 

0. 


Equation  (a)  then  yields  just 

sA^  ■  0, 


I 

I 

I 


I 


whence  we  have 


f 
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_  ^  - ^ - 1 

*  which,  by  (g),  provws  the  theorem,  letting  r  •  r^j  and* 

) 

'noting  (27c)  and  (28). 

» 

This  theorem  can  b#  applied  to  the  short  form  computation 
by  letting 

(29)  Q  -  E,  Q  -  (l,..,l),  and  g  -  -  Xp  .  ' 

•  When,  in  the  course  of  minimizing  it  is  not  possible  to 

reduce  it  under  the  basis  restrictions,  the  hypothesis  of 
Theorem  3  will  be  satisfied,  and  we  will  have 

with  rC  ■  0.  Thus  in  either  the  case  that  C  is  positive 
definite  —  when  necessarily  r  •  0  -  or  in  the  case  that  X-  0 
we  have  “  0,  so  that  the  hypothesis  of  Theorem  2  is 
satisfied,  and  the  terminal  x  solves  the  quadratic  problem. 

Having  reduced  Z  to  zero,  we  maintain  this  in  the  long 
form  and  proceed  to  minimize  —X.  Theorem  3  is  applied  here 
by  letting 

(30)  <J  -  p',  q  -  g  -  0. 

If  the  long  form  recureion  ends  In  a  finite  minimum  for  — 
the  hypothesis  of  Theorem  3  is  satisfied,  whence  we  conclude 
—  \  -  qw  •  rg  •  0; 

—  \  has  in  fact  not  been  reduced.  Two  cases  .are  thus  possible: 
(l)  no  step  lowering  —  A  can  be  taken;  and  (ll)  -X  can  be 


reduced  to  — oo. 
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Casc  ii  H«r«  wt  Must  aakt  utt  of  the  achievable  non-  | 
degeneracy  of  the  eonatrainta  (26)  for  thia  ayatem,  which 

( 

aaaerta  that  nH>n  of  the  varlablea  in  any  aolutlon  are  | 

I 

poaltive.  Since  X"0,  m  of  theae  are  in  u,  and  the  reaaining  ; 

t  I 

n  in  and  x^,  are  empty.  Since  Ar  -0,  Cr  -O,  and, 

i 

from  (27f)  pr  <-l,  we  have  that  for  any  t  I 

» 

1 

(31)  A(x-»'tr')  -  b, 

C(x'ftr  )—  v  +  Au«0.  ! 

It  follows  from  non-degeneracy  that  r^;  for  othenriae  we  ^ 

I 

should  have,  for  sone  t>0,  x-ftr  aatiafylng  (25)  but 

I 

vanishing  in  one  roo'^e  component  than  does  x.  Thus  x^tr  is 
feasible  for  all  t>0,  and 

f(A,x*»-tr')  -  ^px  ♦  ^x'cx  ♦  ^pr*t.  I 

Since  pr'<-l,  f(X,x*t>tr*)  —  oo  as  t->oo  for  any  X>0,  and  j 
the  desired  minimum  is  — oo. 


Case  ii:  The  values  of  X  are  not  bounded.  Since  only  a  ^ 
finite  number  of  bases  is  available,  a  sequence  of  basic 

j  a  s  j 

solutions  (x  ,v  ,u  ,^  ),  l*l,..,g  will  be  produced,  and  ^ 

finally  such  that  (x*-ftx®^^, v®+tv®^^,u*^tu®^^ j 

t)  is  a  solution  for  all  t>p.  Owing  to  the  basis 'restriction 
(l6),  we  will  have  these  relations: 


(32) 


r 
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Olven  now  th#  point 


bting  a  oonvwx  oorabinatlon  of  and  ig  ftasiblt;  and 

it  it  easy  to  ohgck  that,  lotting  v  and  u  ba  reiptctively 
th#  aaao  combinations  of  v^,  and  u^,  u^***^,  thg  resulting 

triple  satisfies  Theorem  3,  so  that  x  yields  the  desired 
minimum.  If  on  the  other  hand  \fi,  the  triple 
X*  >  ( /V. ^  *♦'  +  (A‘^^*)  satiafies 

Theorem  3,  so  that  x^  -♦»  (A'l^®)  !•  the  answer. 


N 
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6.  COMPfTATIOil 

Although  have  prasantad  tha  proetdura  abovt  only  for 
the  oast  of  oonatrainta  of  tha  fora  Ax  >  b,  x  ^  0,  ralying 
on  the  fact  that  all  typae  of  llnaar  Inequality  oonatrainta 
may  be  written  in  thia  fom,  in  praotioal  oomputationa  there 
are  atveral  devieea  which  will  aerve  to  reduce  the  magnitude 
of  the  problem  when  other  typea  of  oonatrainta  are  given.  We 
ahall  give  theae  below  without  proof 0  of  their  effeotiveneaa; 
aueh  proofa  follow  cloaely  along  the  line  of  thoae  of 
Section  4. 

Let  the  oonatrainta  of  the  atated  problem  be 


(The  aecond  and  third  linea  of  (33)  are  the  uaual  fonaulationa 
for  the  oonatrainta  ^  and  The  new  ayatem  of  linear  oon¬ 

atrainta  ( oorreaponding  to  (9)  of  Section  2)  will  be 


*2  yaiP  "1  “aiiP  “3^  A 
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Th«  algorithm  proomada  as  bafora,  with  rule  (I8)  of 
Saotion  3  strangthanad  to: 

(33)  If  basis,  do  not  admit  and 

vioa  varsa;  if  (72)^  Is  In  basis,  do  not  admit  (u2)^«  and 
vioa  versa;  if  (y^)^  is  in  the  basic,  do  not  admit  and 

vice  versa. 

In  this  formulation,  it  is  seen  that  the  number  of 
aquations  in  the  stated  problem  is  augmented  only  by  the  number 
of  non-alaok  variables  in  the  problem.  A  further  reduction  is 
evidently  possible:  since  the  variables  and  u^  are  not 
restricted,  they  could  be  algebraically  eliminated  from  the 
system,  along  with  an  equal  number  of  equations  in  which  they 
have  nonzero  coefficients  (this  reduction  oould  also  have  been 
performed  with  the  u  of  (9)).|  The  eliminations  would  leave  a 
number  of  equations  equal  to  the  total  number  of  components 
of  Xji^,  y2,  and  y^  for  this  generalized  problem,  and  Just  the 
number,  n,  of  components  of  x  in  the  simpler  problem  -  in  any 
case,  the  number  of  inequality  constraints  in  the  original 
problem.  (This  might  seem  odd  if,  for  example,  there  were  no 
inequality  constraints  in  the  stated  problem;  but  then  the 
operations  of  eliminating  the  unrestricted  variables  -  all  the 
variables  -  would  be  precisely  those  of  solving  for  x  and  u  in 
the  classical  Lagrange  Kultiplier  solution.) 

While  the  elimination  is  simple  to  perform,  we  have  not 
employed  it  in  calculation,  for  the  reason  that  it  is  not 
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lik«ly,  in  probitms  whoit  matriots  hav«  many  zero  antrlai,  to 
daoraatf  vazv  graatly  th#  numbar  of  nonzaro  antriaa;  and  it 
is  this  latter  number  which, to  a  oonsidarabla  extant, detanainas 
spaed  of  computation  in  sophisticated  versions  of  the  simplex 
method.  This  must  be  kept  in  mind  when  estimatinf  the  relative 
efficiencies  of  preoedures  of  this  sort;  in  this  procedure,  the 
bulk  of  the  data  consists  of  the  entries  of  C  and  twice  those 
of  A.  In  the  large  probldin  described  below,  these  data  were 
only  1660  in  number,  although  the  resulting  linear  programming 
problem  had  204  equations  and  714  variables. 

A  revision  of  the  SHARI  Linear  Programaing  code  for  the 
IW  704  computer  has  been  made  for  the  solution  of  quadratic 
programming  problems*  This  code  can  be  used  for  either  the 
short  form  or  the  long  form  as  described  above,  or  in  an 
alternate  version  which  first  gets  the  solution  to  the  linear 
problem  obtained  by  dropping  the  quadratic  form,  and  then 
proceeds  to  obtain  the  solutions  for  all  eode  has 

been  used  on  a  variety  of  problems,  the  largest  of  which, 
concerned  with  the  allocation  of  a  strategic  material,  had 
90  constraints  and  192  variables,  78  of  which  were  "slacks 
This  problem  required  359  simplex  method  changes -of -basis 
during  230  minutes  for  the  complete  long  form  solution. 
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